Data and Control Plane Architecture Including Server-Side Triggered Flow Policy Mechanism

ABSTRACT

A data and control plane architecture for network devices. An example system architecture includes a network processing unit implementing one or more data plane operations, and a network device operably coupled to the network processing unit that implements a control plane. In a particular implementation, the network processing unit is configured to process network traffic according to a data plane configuration, and sample selected packets to the network device. The network device processes the sampled packets and adjusts the data plane configuration responsive to the sampled packets. In particular implementations, the control plane and data plane implement, a server-side triggered policy caching mechanism that allows for previous classification policy decisions made for previous data flows to be applied to subsequent new flows.

CROSS-REFERENCE TO RELATED APPLICATIONS AND PATENTS

This application makes reference to the following commonly owned U.S.patent applications and patents, which are incorporated, herein byreference in their entirety for all purposes:

U.S. patent application Ser. No. 08/762,828 now U.S. Pat. No. 5,802,106in the name of Robert L. Packer, entitled “Method for Rapid Data RateDetection in a Packet Communication Environment Without Data RateSupervision;”

U.S. patent application Ser. No. 08/970,693 now U.S. Pat. No. 6,018,516,in the name of Robert L. Packer, entitled “Method for MinimizingUnneeded Retransmission of Packets in a Packet Communication EnvironmentSupporting a Plurality of Data Link Rates;”

U.S. patent application Ser. No. 08/742,994 now U.S. Pat. No. 6,038,216,in the name of Robert L. Packer, entitled “Method for Explicit Data RateControl in a Packet Communication Environment without Data RateSupervision;”

U.S. patent application Ser. No. 09/977,642 now U.S. Pat. No. 6,046,980,in the name of Robert L. Packer, entitled “System for Managing FlowBandwidth Utilization at Network, Transport and Application Layers inStore and Forward Network;”

U.S. patent application Ser. No. 09/106,924 now U.S. Pat. No. 6,115,357,in the name of Robert L. Packer and Brett D. Galloway, entitled “Methodfor Pacing Data Flow in a Packet-based Network;”

U.S. patent application Ser. No. 09/046,776 now U.S. Pat. No. 6,205,120,in the name of Robert L. Packer and Guy Riddle, entitled “Method forTransparently Determining and Setting an Optimal Minimum Required TCPWindow Size;”

U.S. patent application Ser. No. 09/479,356 now U.S. Pat. No. 6,285,658,in the name of Robert L. Packer, en titled “System for Managing FlowBandwidth Utilisation at Network, Transport and Application Layers inStore and Forward Network;”

U.S. patent application Ser. No. 09/198,090 now U.S. Pat. No. 6,412,000,in the name of Guy Riddle and Robert L. Packer, entitled “Method forAutomatically Classifying Traffic in a Packet Communications Network;”

U.S. patent application Ser. No. 10/015,826 now U.S. Pat. No. 7,018,342in the name of Guy Riddle, entitled “Dynamic Tunnel Probing in aCommunications Network;”

U.S. patent application Ser. No. 10/039,992 now U.S. Pat. No. 7,032,072,in the name of Michael J. Quinn and Mary L. Later, entitled “Method andApparatus for Fast Lookup of Related Classification Entities in aTree-Ordered Classification Hierarchy;”

U.S. patent application Ser. No. 10/155,936 now U.S. Pat. No. 6,591,299,in the name of Guy Riddle, Robert L. Packer, and Mark Hill, entitled“Method For Automatically Classifying Traffic With Enhanced Hierarchy InA Packet Communications Network;”

U.S. patent application Ser. No. 09/206,772, now U.S. Pat. No.6,456,360, in the name of Robert L. Packer, Brett D. Galloway and TedThi, entitled “Method for Data Rate Control for Heterogeneous or PeerInternetworking;”

U.S. patent application Ser. No. 09/198,051, in the name of Guy Riddle,entitled “Method for Automatically Determining a Traffic Policy in aPacket Communications Network;”

U.S. patent application Ser. No. 09/966,538, in the name of Guy Riddle,entitled “Dynamic Partitioning of Network Resources;”

U.S. patent application Ser. No. 11/053,596 in the name of Azeem Feroz,Wei-Lung Lai, Roopesh R. Varier, James J. Stabile, and Jon Eric Okholm,entitled “Aggregate Network Resource Utilization Control Scheme;”

U.S. patent application Ser. No. 10/108,085, in the name of Wei-LungLai, Jon Eric Okholm, and Michael J. Quinn, entitled “Output SchedulingData Structure Facilitating Hierarchical Network Resource AllocationScheme;”

U.S. patent application Ser. No. 10/236,149, in the name of BrettGalloway and George Powers, entitled “Classification Data Structureenabling Multi-Dimensional Network Traffic Classification and ControlSchemes;”

U.S. patent application Ser. No. 10/334,467, in the name of Mark Hill,entitled “Methods, Apparatuses and Systems Facilitating Analysis of thePerformance of Network Traffic Classification Configurations;”

U.S. patent application Ser. No. 10/453,345, in the name of ScottHankins, Michael R. Morford, and Michael J. Quinn, entitled “Flow-BasedPacket Capture;”

U.S. patent application Ser. No. 10/676,383 in the name of Guy Riddle,entitled “Enhanced Flow Data Records Including Traffic Type Data;”

U.S. patent application Ser. No. 10/720,329, in the name of Weng-ChinYung, Mark Hill and Anne Cesa Klein, entitled “Heuristic BehaviorPattern Matching of Data Flows in Enhanced Network TrafficClassification.”

U.S. patent application Ser. No. 10/812,198 in the name of MichaelRobert Morford and Robert E. Purvy, entitled “Adaptive,Application-Aware Selection of Differentiated Network Services;”

U.S. patent application Ser. No. 10/843,185 in the name of Guy Riddle,Curtis Vance Bradford and Maddie Cheng, entitled “Packet Load Shedding;”

U.S. patent application Ser. No. 10/917,952 in the name of Weng-ChinYung, entitled “Examination of Connection Handshake to EnhanceClassification of Encrypted Network Traffic;”

U.S. patent application Ser. No. 10/938,435 in the name of Guy Riddle,entitled “Classification and Management of Network Traffic Based onAttributes Orthogonal to Explicit Packet. Attributes;”

U.S. patent application Ser. No. 11/019,501 in the name of SureshMuppala, entitled “Probing Hosts Against Network Application Profiles toFacilitate Classification of Network Traffic;”

U.S. patent application Ser. No. 11/027,744 in the name of Mark Urban,entitled “Adaptive Correlation of Service Level Agreement and NetworkApplication Performance;” and

U.S. patent application Ser. No. 11/241,007 in the name of Guy Riddle,entitled “Partition Configuration and Creation Mechanisms for NetworkTraffic Management Devices.”

TECHNICAL FIELD

This disclosure relates generally to network application trafficmanagement.

BACKGROUND

Enterprises have become increasingly dependent on computer networkinfrastructures to provide services and accomplish mission-criticaltasks. Indeed, the performance, security, and efficiency of thesenetwork infrastructures have become critical as enterprises increasetheir reliance on distributed computing environments and wide areacomputer networks. To that end, a variety of network devices have beencreated to provide data gathering, reporting, and/or operationalfunctions, such as firewalls, gateways, packet capture devices,bandwidth management devices, application traffic monitoring devices,and the like. For example, the TCP/IP protocol suite, which is widelyimplemented throughout the world-wide data communications networkenvironment called the Internet and many wide and local area networks,omits any explicit supervisory function over the rate of data transportover the various devices that comprise the network. While there arecertain perceived advantages, this characteristic has the consequence ofjuxtaposing very high-speed packets and very low-speed packets inpotential conflict and produces certain inefficiencies. Certain loadingconditions degrade performance of networked applications and can evencause instabilities which could lead to overloads that could stop datatransfer temporarily.

In response, certain data flow rate control mechanisms have beendeveloped to provide a means to control and optimize efficiency of datatransfer as well as allocate available bandwidth among a variety ofbusiness enterprise functionalities. For example, U.S. Pat. No.6,038,216 discloses a method for explicit data rate control in apacket-based network environment without data rate supervision. Datarate control directly moderates the rate of data transmission from asending host, resulting in just-in-time data transmission to controlinbound traffic and reduce the inefficiencies associated with droppedpackets. Bandwidth management devices allow for explicit data ratecontrol for flows associated with a particular traffic classification.For example, U.S. Pat. No. 6,412,000, above, discloses automaticclassification of network traffic for use in connection with bandwidthallocation mechanisms. U.S. Pat. No. 6,046,980 discloses systems andmethods allowing for application layer control of bandwidth utilizationin packet-based computer networks. For example, bandwidth managementdevices allow network administrators to specify policies operative tocontrol and/or prioritize the bandwidth allocated to individual dataflows according to traffic classifications. In addition, networksecurity is another concern, such as the detection of computer viruses,as well as prevention of Denial-of-Service (DoS) attacks on, orunauthorized access to, enterprise networks. Accordingly, firewalls andother network devices are deployed at the edge of such networks tofilter packets and perform various operations in response to a securitythreat. In addition, packet capture and other network data gatheringdevices are often deployed at the edge of, as well as at other strategicpoints in, a network to allow network administrators to monitor networkconditions.

Enterprise network topologies can span a vast array of designs andconnection schemes depending on the enterprise's resource requirements,the number of locations or offices to connect, desired service levels,costs and the like. A given enterprise often must support multiple LANor WAN segments that support headquarters, branch offices and otheroperational and office facilities. Indeed, enterprise network designtopologies often include multiple, interconnected LAN and WAN segmentsin the enterprise's intranet, and multiple paths to extranets and theInternet. Enterprises that cannot afford the expense of privateleased-lines to develop their own WANs, often employ frame relay, orother packet switched networks, together with Virtual Private Networking(VPN) technologies to connect private enterprise sites via a serviceprovider's public network or the Internet. Some enterprises also use VPNtechnology to create extranets with customers, suppliers, and vendors.These network topologies often require the deployment of a variety ofnetwork devices at each remote facility. In addition, some networksystems are end-to-end solutions, such as application traffic optimizersusing compression tunnels, requiring network devices at each end of acommunications path between, for example, a main office and a remotefacility.

Many of the network, devices discussed above are typically deployed atstrategic locations in the network topology such that all or nearly allnetwork traffic flows through them. For example, firewall and intrusiondetection systems are typically deployed at the edges of a networkdomain to filter incoming and outgoing traffic. Similarly, bandwidthmanagement systems are typically deployed between a network and anaccess link to allow for more direct control of access link utilization.Given that these network devices may process large amounts of networktraffic (especially during peak load conditions), they must possesssufficient computing resources to provide for sufficient performance andthroughput. If the network device becomes a bottleneck, latencyincreases and degrades network application performance. Still further,the processes and functions performed by these network devices arebecoming more complex and, thus, require higher processing power thanprevious generation systems. Indeed, bandwidth management systems, forexample, have evolved to include complex packet inspection,classification and control mechanisms.

In some previous approaches to increasing the performance of networkdevices, vendors have simply relied on more powerful processors,frequently turning to customized hardware solutions. This approach,however, is inherently limited to the capability of the custom hardware.Custom hardware solutions also require increased development costs andlong lead times, as well as limited flexibility for correcting bugs andadapting to changing customer requirements. In addition, while somenetwork device manufactures have turned to systems with multipleprocessors, they have not addressed the challenges posed, by QoS andother devices that employ stateful or flow-aware inspection,classification and control mechanisms.

DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram illustrating a computer networkenvironment, in which implementations of the invention may operate.

FIGS. 2A and 2B are schematic diagrams illustrating the interconnectionsbetween a network application traffic management device and a networkprocessing unit according to particular implementations of theinvention.

FIG. 3 is a functional block diagram that shows the components andsystem architecture of a network application traffic management deviceand a network processing unit according to one particular implementationof the invention.

FIG. 4A is a schematic diagram illustrating logical processing modulesof an application traffic management device according to one particularimplementation of the invention.

FIG. 4B is a functional block diagram illustrating a process flow,according to one implementation of the present invention, among varioushardware and software modules of a network application trafficmanagement unit.

FIG. 5A is a flow chart setting forth a method, according to onepossible implementation of the invention, directed to processing flowsat a network processing unit.

FIG. 5B is a flow chart illustrating another example method directed toprocessing flows at a network processing unit.

FIG. 5C is a flow chart providing an example method directed toprocessing control messages at a network processing unit.

FIG. 6 is a state diagram, according to a particular implementation ofthe invention, directed, to controlling the sampling of packets, and therate at which packets are sampled, to a network application trafficmanagement device.

FIG. 7 is a flow chart illustrating a method directed to samplingpackets.

FIG. 8 is a flow chart setting forth an example method, directed tosampling packets to a control plane.

FIG. 9 is a flow chart illustrating an example method directed toprocessing received packets sampled by a data plane.

DESCRIPTION OF EXAMPLE EMBODIMENTS A. Overview

The present invention provides methods, apparatuses and systems directedto a network device system architecture that increases throughput ofdevices that process network traffic. In a particular implementation, anexample system architecture includes a network device implementing acontrol plane, that is operably coupled to a network processing unitimplementing one or more data plane operations. In a particularimplementation, the network processing unit is configured to processnetwork traffic according to a data plane configuration, and sampleselected packets of a data, flow to the network device. The controlplane of the network device processes the sampled packets and adjuststhe data plane configuration, responsive to the sampled, packets of thedata flow. In particular implementations, the control plane and dataplane implement a server-side triggered policy caching mechanism thatallows for previous classification policy decisions made for previousdata flows to be applied to subsequent new flows.

In particular implementations, the present invention is directed tomethods, apparatuses and systems that use fast network processors toaccelerate the operation of existing slower network device hardwareplatforms. As described herein, the architecture allows the bulk ofnetwork traffic processing to be offloaded to the fast network processorinstead of the network device. In a particular implementation, thepresent invention provides a cost effective solution to increasing thethroughput of existing hardware with little to no modification to theexisting hardware and minimal changes to software or firmware with theuse of an external appliance or device that implements a data plane canbe used to increase the throughput of existing hardware with little tono modification to the existing hardware and minimal changes to softwareor firmware to implement control plane operations.

In the following description, specific details are set forth in order toprovide a thorough understanding of particular implementations of thepresent invention. Other implementations of the invention may bepracticed without some or all of specific details set forth below. Insome instances, well known structures and/or processes have not beendescribed in detail so that the present invention is not unnecessarilyobscured.

A.1. Network Environment

FIG. 1 illustrates, for didactic purposes, a network 50, such as widearea network, interconnecting a first network 40 a, supporting a centraloperating or headquarters facility (for example), and a second network40 b, supporting a branch office facility (for example). In oneimplementation, network 50 may include a MPLS VPN network coreinterconnecting networks 40 a and 40 b. Network 50 may also be operablyconnected to other networks associated with the same administrativedomain as networks 40 a, 40 b, or a different administrative domain.Furthermore, network 50 may allow access to a variety of hosts over theInternet, as well. As FIG. 1 shows, the first network 40 a interconnectsseveral hosts or end systems 42, including personal computers andservers, and provides access to resources operably connected to computernetwork 50 via router 22 and access link 21. Access link 21 is aphysical and/or logical connection between two networks, such ascomputer network 50 and network 40 a. The computer network environment,including network 40 a and network 50 is a packet-based communicationsenvironment, employing TCP/IP protocols (for example), and/or othersuitable protocols, and has a plurality of interconnected digital packettransmission stations or routing nodes. First network 40 a, and network40 b, can each be a local area network, a wide area network,combinations thereof, or any other suitable network.

As FIG. 1 illustrates, network devices 30, in one implementation, aredeployed at the respective edges of networks 40 a and 40 b. In aparticular implementation, network devices 30 are network applicationtraffic management devices operative to manage network applicationtraffic. As discussed below, network application traffic managementdevices 30 may include a number of different functional modules, such ascompression modules, tunneling modules, rate control modules, gatewaymodules, protocol acceleration modules, and the like. In addition,network application traffic management devices 30 may include functions,such as compression and/or tunneling, where cooperation with a remotedevice (such as another network application traffic management device)is required, while also performing other functions that can be performedindependently. However, the control and data plane system architectureaccording to the present invention can be integrated into a variety ofnetwork devices, such as proxies, firewalls, intrusion detectionsystems, packet capture or network monitoring equipment, VPN servers,web services network gateways or brokers, and the like.

A.2. Example System Architecture

FIG. 2A illustrates an example system architecture of networkapplication traffic management device 30. In the implementation shown,network application traffic management device 30 comprises anapplication traffic management unit 200 and a network processing unit300. Network application traffic management unit 200 may be implementedusing existing hardware device platforms and functionality with smallsoftware modifications to interact with network processing unit 300.Network processing unit 300 interconnects with application trafficmanagement unit as shown, and is further operably connected to network50 and LAN 40 a. In the implementation shown, network processing unit300 is a discrete processing unit that does not share memory withnetwork application traffic management unit 200, instead beinginterconnected via one or more network interfaces. In otherimplementations, the interconnections between network processing unit300 and network application traffic management unit 200 can beimplemented using other suitable interface technologies, such as othernetwork interface technologies, and bus interfaces (e.g., PeripheralComponent Interconnect (PCI) Interfaces, and Industry StandardArchitecture (ISA) interfaces). Furthermore, network application trafficmanagement unit 200 and network processing unit 300 may be directlyinterconnected to each other with only network cabling extending betweenthe packet interfaces. In another implementation, the packet, interfacesof the network application traffic management unit 200 and networkprocessing unit 300 may be connected to an Ethernet switch or othernetwork fabric. The devices could use layer 2 or 3 tunneling protocolsto transmit sampled packets to each other.

As FIG. 2A illustrates network application traffic management unit 200comprises packet interfaces 216 a and 216 b, while network processingunit 300 comprises packet interfaces 320 a-d. In one implementation,packet interfaces comprise Ethernet interfaces including MAC layerfunctionality. However, other network interfaces can be used dependingon the network environment. Generally, packets received from network 50on packet interface 320 a are transmitted to network 40 a from packetinterface 320 b, and vice versa. Network processing unit 300, generallyspeaking, may be configured to implement one or more data planeoperations on the network traffic transmitted between network 50 andnetwork 40 a according to a data plane configuration. As discussed inmore detail below, network processing unit 300 is configured to receivepackets from network 50 or 40 a and selectively sample received packetsto application traffic management unit 200 for processing. In oneimplementation, packets received on packet interface 320 a, whensampled, are transmitted from packet interface 320 c to packet interface216 a of network application traffic management unit 200. Networkprocessing unit 300, in one implementation, then processes the packetwithout waiting for a response from network application trafficmanagement unit 200. In a particular implementation, the networkprocessing unit 300, given that it processes packets according to itscurrent data plane configuration, can continue to process packets evenwhen the control plane crashes and/or reboots. In other implementations,network application traffic management unit 200 can be configured toforward or emit some packets passed to it instead of the networkprocessing unit 300. In addition, network application traffic managementunit 200 may also be configured to emit probe messages and othermessages directed to device discovery, network management, and the like,directly as well. Application traffic management unit 200 performsvarious control plane operations on sampled packets, such as packetclassification, policy identification, and the like. Responsive to oneor more received sample packets, network application traffic managementunit 200 may transmit one or more control messages to network processingunit 300 operative to cause changes to the data plane configuration ofthe network processing unit. For example, network application trafficmanagement unit 200 may receive sampled packets of a data flow, andclassify them to identify one or more policies or controls. Networkapplication traffic management unit 200 may then transmit a controlmessage identifying one or more traffic classification or service typesand one or more policies to be applied to packets of the data flow.Network processing unit 300 may then apply the one or more policies tosubsequent packets of the same data flow.

Other implementations are possible. For example, network applicationtraffic management unit 200 and network processing unit 300 could beconnected using a single pair of packet interfaces. In otherimplementations, network application traffic management unit 200 andnetwork processing unit 300 could be connected with additional packetinterfaces than that shown in FIG. 2A. For example, a packet interfacepair between network application traffic management unit 200 and networkprocessing unit 300 could be configured for each packet interface ofnetwork processing unit 300 that is connected to a network, such as aninternal LAN or a WAN. As FIG. 2B shows, in another implementation,network processing unit 300 could be configured with multiple packetinterfaces 320 a, 320 b, 320 e, and 320 f for communication withrespective networks 50, 40 a, 51 & 41 a, and multiplex packetstransmitted to an network application traffic management unit 200 over asmaller number of packet interfaces 320 c and 320 d.

In yet another implementation, a single network application trafficmanagement unit 200 can be connected to multiple network processingunits 300 disposed at various points in a network environment. Forexample, two network processing units 300 could be deployed on separateaccess links, and communicably coupled to a single network applicationtraffic management unit 200. Conversely, a single network processingunit 300 could be operably coupled to multiple application trafficmanagement units 200. In one such implementation, the network processingunit can be configured to ensure that packets of the same flow aretransmitted to the same network application traffic management 200.

A.2.a. Network Application Traffic Management Unit

While network application traffic management unit 200 may be implementedin a number of different hardware architectures, some or all of theelements or operations thereof may be implemented using a computingsystem having a general purpose hardware architecture such as the one inFIG. 3. In one implementation, network application traffic managementunit 200 comprises a processor 202, a cache memory 204, and one or moresoftware applications and drivers directed to the functions describedherein. In one implementation, network application traffic managementunit 200 includes a high performance input/output (I/O) bus 206 and astandard I/O bus 208. A host bridge 210 couples processor 202 to highperformance I/O bus 206, whereas I/O bus bridge 212 couples the twobuses 206 and 208 to each other. A system memory 214 and one or morenetwork/communication interfaces 216 couple to bus 206. Mass storage 218and I/O ports 220 couple to bus 208. Collectively, these elements areintended to represent a broad category of computer hardware systems,including but not limited to general purpose computer systems based onthe Pentium® processor manufactured by Intel Corporation of Santa Clara,Calif., as well as any other suitable processor.

Network interface 216 c provides communication between networkapplication traffic management unit 200 and a network through which auser may access management or reporting functions. Mass storage 218provides permanent storage for the data and programming instructions toperform the above described functions implemented in the systemcontroller, whereas system memory 214 (e.g., DRAM) provides temporarystorage for the data and programming instructions when executed byprocessor 202. I/O ports 220 are one or more serial and/or parallelcommunication ports that provide communication between additionalperipheral devices, which may be coupled to network application trafficmanagement unit 200.

Network application traffic management unit 200 may include a variety ofsystem architectures, and various components of network applicationtraffic management unit 200 may be rearranged. For example, cache 204may be on-chip with processor 202. Alternatively, cache 204 andprocessor 202 may be packed together as a “processor module,” withprocessor 202 being referred to as the “processor core.” Furthermore,certain implementations of the present invention may not require norinclude all of the above components. For example, the peripheral devicesshown coupled to standard. I/O bus 208 may couple to high, performanceI/O bus 206. In addition, in some implementations only a single bus mayexist, with the components of network application traffic managementunit 200 being coupled to the single bus. Furthermore, networkapplication traffic management unit 200 may include additionalcomponents, such as additional processors, storage devices, or memories.

The operations of the network application traffic management unit 200described herein are implemented as a series of software routines (seeFIGS. 4A and 4 b) hosted by network application traffic management unit200. These software routines comprise a plurality or series ofinstructions to be executed by a processor in a hardware system, such asprocessor 202. Initially, the series of instructions are stored on astorage device, such as mass storage 218. However, the series ofinstructions can be stored on any suitable storage medium, such as adiskette, CD-ROM, ROM, EEPROM, etc. Furthermore, the series ofinstructions need not be stored locally, and could be received from aremote storage device, such as a server on a network, vianetwork/communication interface 216 c. The instructions are copied fromthe storage device, such as mass storage 218, into memory 214 and thenaccessed and executed by processor 202.

An operating system manages and controls the operation of networkapplication traffic management unit 200, including the input and outputof data to and from software applications (not shown). The operatingsystem provides an interface between the software applications beingexecuted, on the system and the hardware components of the system.According to one embodiment of the present invention, the operatingsystem is a realtime operating system, such as PSOS, or LINUX. In otherimplementations, the operating system may be the Windows®95/98/NT/XP/Vista operating system, available from Microsoft Corporationof Redmond, Wash. However, the present invention may be used with othersuitable operating systems, such as the Apple Macintosh OperatingSystem, available from Apple Computer Inc. of Cupertino, Calif., UNIXoperating systems, and the like.

FIG. 4B provides an illustration of the components and functionalmodules, and data structures, relevant, to how packets are processed bynetwork application traffic management unit 200. As FIG. 4B illustrates,network application traffic management unit 200 includes inside NICreceive ring 81 a, outside NIC receive ring 81 b, inside NIC receivequeue 84 a, outside NIC receive queue 84 b, NIC driver 83, and packetbuffer 82. Packet buffer 82 is operative to store packets received atpacket interfaces 216 a, 216 b. To summarize the operations associatedwith receiving and ultimately processing packets, network interface 216a, for example, receives and stores a packet in packet buffer 82.Network interface 216 a also maintains a pointer to the packet in insideNIC receive ring 81 a. As discussed more fully below, NIC driver 83determines whether to queue or otherwise retain the packet, or todiscard, it. In one implementation, NIC driver 83, operating at periodicinterrupts, writes pointers out of inside NIC receive ring 81 a and intoinside NIC receive queue. Network application traffic management unit200 operates substantially identically for packets received at outsidepacket interface 216 b. Network device application 75, in oneimplementation, operates on packets stored in packet buffer 82 byaccessing the memory address spaces (pointers) to the packets in insideNIC receive queue 84 a and outside NIC receive queue 84 b. In oneimplementation, a sampled packet received at inside packet interface 216a is usually dropped, after processing by network device application 75,as opposed to being transmitted from outside packet interface 216 b. Therings and other data structures supporting the transmission of packetsfrom network interfaces 216 a, 216 b are not shown.

In one implementation, packet buffer 82 comprises a series of fixed-sizememory spaces for each packet (e.g., 50,000 spaces). In otherimplementations, packet buffer 82 includes mechanisms allowing forvariable sized memory spaces depending on the size of the packet. InsideNIC receive ring 81 a is a circular queue or ring of memory addresses(pointers) corresponding to packets stored in packet buffer 82. In oneimplementation, inside NIC receive ring 81 a includes 256 entries;however, the number of entries is a matter of engineering and designchoice. In one implementation, each entry of inside NIC receive ring 81a includes a field for a memory address, as well as other fields forstatus flags and the like. For example, one status flag indicateswhether the memory address space is empty or filled with a packet.Inside NIC receive ring 81 a also maintains head and tail memoryaddresses, as described below. In one implementation, packet interface216 a also maintains the head and tail memory address spaces in itsregisters. The head memory address space corresponds to the nextavailable memory space in packet buffer 82 to which the next packet isto be stored. Accordingly, when packet interface 216 a receives apacket, it checks the head address register to determine where in thesystem memory reserved for packet buffer 82 to store the packet. Afterthe packet is stored, the status flag in the ring entry is changed tofilled. In addition, the system memory returns a memory address forstoring the next received packet, which is stored in the next entry ininside NIC receive ring 81 a, in addition, the head address register isadvanced to this next memory address. The tail memory address spacecorresponds to the earliest received packet which has not been processedby NIC driver 83. In one implementation, packet interface 216 a alsomaintains a copy of inside NIC receive ring 81 a in a memory unitresiding on the network interface hardware itself. In oneimplementation, packet interface 216 a discards packets when inside NICreceive ring 81 a is full i.e., when the tail and head memory addressesare the same.

As discussed above, NIC driver 83 is operative to read packet pointersfrom inside NIC receive ring 81 a to inside NIC receive queue 84 a. Inone implementation, NIC driver 83 operates on inside NIC receive ring 81a by accessing the tail memory address to identify the earliest receivedpacket. To write the packet in the inside NIC receive queue 84 a, NICdriver 83 copies the memory address into inside NIC receive queue, setsthe status flag in the entry in inside NIC receive ring 81 acorresponding to the fail memory address to empty, and advances the tailmemory address to the next entry in the ring. NIC driver 88 can discarda packet, by simply dropping it from inside NIC receive ring 81 a, andnot writing it into inside NIC receive queue 84 a. As discussed morefully below, this discard operation may be performed in connection withrandom early drop mechanisms, or the load shedding mechanisms, accordingto those described in U.S. application Ser. No. 10/843,185, incorporatedby reference herein. Still further, NIC driver 83, in oneimplementation, is a software module that operates at periodicinterrupts to process packets from inside NIC receive ring 81 a toinside NIC receive queue 84 a. At each interrupt, NIC driver 83 canprocess all packets in receive ring 81 a or, as discussed more fullybelow, process a limited number of packets. Furthermore, as discussedmore fully below, a fairness algorithm controls which of inside NICreceive ring 81 a and outside NIC receive ring 81 b to process first ateach interrupt.

In one implementation, inside NIC receive ring 81 a, outside NIC receivering 81 b, inside NIC receive queue 84 a, outside NIC receive queue 84b, and packet buffer 82 are maintained in reserved spaces of the systemmemory of network application traffic management unit 200. As discussedabove, network device application 75, operating at a higher level,processes packets in packet buffer 82 reading packet pointers fromreceive queues 84 a, 84 b. The system memory implemented in networkapplication traffic management unit 200, in one embodiment, includes oneor more DRAM chips and a memory controller providing the interface, andhandling the input-output operations, associated with storing data inthe DRAM chip(s). In one implementation, the hardware in networkapplication traffic management unit 200 includes functionality allowingfirst and second network interfaces 216 a, 216 b to directly accessmemory 82 to store inbound packets received at the interfaces in packetbuffer. For example, in one implementation, the system chip setassociated with network application traffic management unit 200 caninclude a Direct Memory Access (DMA) controller, which is a circuit thatallows for transfer of a block of data from the buffer memory of anetwork interface, for example, directly to memory 82 without CPUinvolvement. A variety of direct memory access technologies andprotocols can be used, such as standard DMA, first-party DMA (busmastering), and programmed I/O (PIO). In one implementation, eachnetwork interface 216 a and 216 b is allocated a DMA channel to thememory 82 to store packets received at the corresponding interfaces.

In addition, the system chip set of network application trafficmanagement unit 200, in one implementation, further includes aninterrupt controller to receive and prioritize interrupt requests (IRQs)transmitted by devices over the system bus. Network application trafficmanagement unit 200, in one implementation, further includes aninterrupt timer that periodically transmits an interrupt signal to theinterrupt controller. In one implementation, the interrupt controller,after receiving the periodic interrupt signal, dedicates the CPU andother resources to NIC driver 83 to process received packets asdiscussed above. In one implementation, the interrupt timer transmitsinterrupt signals every 50 microseconds; of course, this interval is amatter of engineering or system design choice. In certainimplementations of the present invention, network interfaces 216 a, 216b can transmit demand-based interrupts after packets have arrived.

FIG. 4A is a block diagram illustrating functionality, according to oneembodiment of the present invention, included in network applicationtraffic management unit 200. In one embodiment, network application 75of network application traffic management unit 200 comprises packetprocessor 131, data plane interface module 132, measurement engine 140,traffic classification engine 137, management information base (MIB)138, and administrator interface 150. The co-pending and commonly ownedpatents and patent applications identified above describe variousfunctions and operations that can be incorporated into networkapplication traffic management unit 200. Packet processor 131 isoperative to detect new data flows and construct, data structuresincluding attributes characterizing the data flow. Data plane interfacemodule 132 is operative to generate control messages and transmit themto network processing unit 300, as well as receive packets (controlmessages and sampled packets) from network processing unit 300 andselectively forward sampled packets to packet processor 131. Trafficclassification, engine 137 is operative to analyze data flow attributesand identify traffic classes corresponding to the data flows. In oneembodiment, traffic classification engine 137 stores traffic classes, inassociation with pointers to bandwidth utilization controls or pointersto data, structures defining such bandwidth utilization controls.Management, information base 138 is a database of standard and extendednetwork objects related to the operation of network application trafficmanagement unit 200. Measurement engine 140 maintains measurement datarelating to operation of network application traffic management unit 200to allow for monitoring of bandwidth utilization across access link 21with respect to a plurality of bandwidth utilization and other networkstatistics on an aggregate and/or per-traffic-class level. Networkapplication traffic management unit 200, in one embodiment, furtherincludes a persistent data store (not shown), such as a hard disk drive,for non-volatile storage of data.

Administrator interface 150 facilitates the configuration of networkapplication traffic management unit 200 to adjust or change operationaland configuration parameters associated with the device. For example,administrator interface 150 allows administrators to select identifiedtraffic classes and associate them with bandwidth utilization controls(e.g., a partition, a policy, etc.). Administrator interface 150, in oneimplementation, also displays various views associated with a trafficclassification scheme and allows administrators to configure or revisethe traffic classification scheme. Administrator interface 150 can be acommand line interface or a graphical user interface accessible, forexample, through a conventional browser on client device 42. Inaddition, since in one implementation, network processing unit 300 maynot be a network addressable device and only responds to controlmessages transmitted from network application traffic management unit200, administrator interface 150 provides a unified user interface fornetwork application traffic management unit 200 and network processingunit 300 in the aggregate.

As disclosed in U.S. application Ser. No. 10/843,185, the number ofpackets in the inside or outside NIC receive queues 84 a, 84 b can bemonitored to signal a possible overload condition. That is when thenumber of packets in one of the queues exceeds a threshold parameter,network application traffic management unit 200 may perform one or moreactions. In one implementation, network application traffic managementunit 200 may transmit a message to network processing unit 300 signalingthat it is at or near an overload state. As described in more detailbelow, network processing unit 300 responsive to such a message may stopsampling packets to network application traffic management unit 200 orreduce the rate at which packets are sampled. Still further, asdescribed in U.S. application Ser. No. 10/843,185, network applicationtraffic management unit 200 may access host database 134 to comparecertain observed parameters corresponding to the source hosts identifiedin received packets, and compare them against corresponding thresholdvalues to determine whether to discard received packets. For example, ahost identified as being part of a Denial-of-Service attack may bedeemed a “bad host.” In one implementation, network application trafficmanagement unit 200 may transmit control messages to network processingunit 300 directing it to drop packets from an identified bad host.

When network application traffic management unit 200 operates withoutnetwork processing unit 300 it generally operates to receive packets ata first interface (e.g., packet interface 218 a), process the packets,and emit the packets at a second interface (e.g., packet interface 216a), or vice versa. When configured to operate in connection with networkprocessing unit 300, however, network application traffic managementunit 200 is configured to receive and process the packets sampled to it,but to drop the packets instead of emitting them. As part of thisprocess, network application traffic management unit 200, in oneimplementation, receives a sampled packet, processes the packet, and maytransmit one or more control messages to network processing unit 300indicating how subsequent packets of a data flow should be handled.

A.2.b. Network Processing Unit

FIG. 3 also illustrates an example system architecture for a networkprocessing unit 300 according to a particular implementation of theinvention. In one implementation, network processing unit 300 includes anetwork processor having one to multiple processing cores. A networkprocessor is an integrated circuit which has a processing feature setspecifically adapted to the networking application domain. In oneparticular implementation, the network processor is a softwareprogrammable device allowing the feature set to be used in a flexiblemanner. One example of a network processor that can be used in thepresent invention is the Octeon™ Plus CN58XX 4 to 16-Core MIPS64-BasedSoCs offered by Cavium Networks of Mountain View, Calif. Of course, anyother suitable network processor can be used. In the implementationshown in FIG. 3, network processing unit 300 comprises processor 302(comprising one to a plurality of processing cores), cache 304 (e.g., L2cache shared among the processing cores), memory controller 306 andrandom access memory (RAM) 308. Network processing unit 300, in oneimplementation, further comprises packet input module 310, I/O bridge312, packet output module 314, TCP unit 316, scheduler 318 (in oneimplementation, for packet scheduling and queuing-based Quality ofService (QoS)), and packet interfaces 320 a-d. Network processing unit300 may also include other functional modules such as a regularexpression unit for string matching, a compression/decompression unitfor accelerating network traffic, and an encryption unit. Still further,network processing unit 300, in some implementations, may includemultiple network processors each having multiple processing cores.

Although not illustrated, in one implementation, network processing unit300 may also include a power supply, RJ-45 or other physical connectors,and a chassis separate from network application traffic management unit200. For example, as discussed above, network processing unit 300 may bea separate physical unit in the form factor of a 1 U or 2 U appliance.The network processing unit 300 may be used to accelerate and enhancethe throughput of an existing network application traffic managementdevice, such as network application traffic management unit 200. In oneimplementation, without network processing unit 300, application trafficmanagement unit 200 would be directly connected to the network pathsegment between network 50 and network 40 a. For example, packetinterface 216 a would be operably connected to network 50, while packetinterface 216 b would be operably connected, to network 40 a. Toincrease throughput, however, network processing unit 300 may beinterconnected as shown in FIGS. 2 and 3. As discussed in more detailbelow, network processing unit 300 can be configured to perform variousdata plane operations, and to selectively forward packets to applicationtraffic management unit 200. In one possible configuration, applicationtraffic management unit 200 performs higher-level processing of packetsof respective data flows to classify the data flows and identify one ormore policies to be applied to the data flows. Throughput can beincreased, due to the capabilities of the network processing unit 300 toperform data plane operations on packets at wireline or near wirelinespeeds, and that network application traffic management unit 200 seesonly a subset of all packets traversing networks 40 a and 50.Accordingly, implementations of the invention provide for an inexpensiveand easily deployable solution that accelerates the performance ofexisting hardware and allows an end-user to preserve investments inexisting hardware platforms. In one such implementation, the networkprocessing unit 300 can be distributed with a computer-readable media,such as optically or magnetically recorded disks or tapes, that includeone or more software modules that, when installed, modify the operationof the network application traffic management unit 200 to interact withthe network processing unit 300 as described herein. Thecomputer-readable media may also include a copy of firmware for thenetwork processing unit 300. In one implementation, network applicationtraffic management unit 200 can store the firmware and provide it tonetwork processing unit 300 during a configuration session. In otherimplementations, however, the network processing unit 300 and networkapplication traffic management unit 200 may be incorporated into thesame chassis.

B. Control Messages

As described herein, network application traffic management unit 200(Control Plane) and network processing unit 300 (Data Plane) implement atwo-way message path by which network application traffic managementunit 200 directs network processing unit 300 which policies should beapplied to the data flows traversing it. In a particular implementation,network processing unit 300 also returns network statistics, such asMeasurement Samples, to be integrated into the measurement and reportingfunctionality of measurement engine 140 of network application trafficmanagement unit 200. The Control Plane makes the flow decision aftercompleting classification of the flow, including peeking at the datapackets as necessary and consulting the policies stored in the trafficclassification engine 137. Example network traffic classificationmechanisms are described in U.S. application Ser. No. 11/019,501, aswell as other patents and patent applications identified above. Stillfurther, as discussed in more detail below, the Control Plane may directthe Data Plane to cache the traffic classification and policy decisionsmade in connection with a given data flow, and apply the same policiesto subsequent data flows that include a set of matching attributes, suchas destination network address, port and protocol identifiers.

Control messages between the network application traffic management unit200 and network processing unit 300, in one implementation, use aspecific VLAN to facilitate identification of control messages and othercommunications between them. In some implementations, VLAN tagging isnot employed. Flow Information Messages have the same IP and TCP/UDPprotocol headers as the flow they refer to in order to get the sametuple hash from the network processor hardware. Alternatively, flowinformation messages can be encapsulated in IP-in-IP or Generic RoutingEncapsulation (GRE) or other tunneling protocols. Other control messagesuse specific addresses for the network application traffic managementunit 200 and network processing unit 300. These are local to the twounits (in one implementation, chosen from the 127 class A address range)and need no configuration.

In a particular implementation, there are 5 types of control messagesfrom the Control Plane to the Data Plane, and 3 types of controlmessages in the reverse direction. The first message sent to the ControlPlane is the SizingData message describing one or more attributes ofvarious operational data structures, such as the sizes of tables.Partitioning messages are sent to describe the configuration ofpartitions, and any subsequent changes. A FlowInfo message is sent whennetwork application traffic management unit 200 decides on the partitionand policy to apply to a flow. Two message types, the OverloadStatus andthe BadHostInfo inform the Data Plane when the network applicationtraffic management unit 200 enters or leaves an overloaded condition andof any hosts the Load Shedding feature decides are behaving badly.

The three types of messages sent from tire Data Plane to the ControlPlane are the ReTransmitRequest to recover from possible lost messagesor to resynchronize, the Measurements ample message to transmitmeasurement samples for the configured traffic classes and partitions,and the LittleNote to transmit status messages to be logged.

Other message types may also be implemented for different functions. Forexample, one or more message types may be configured for compressionfunctions, such as a message for setting up Layer 3 tunnels with remotenodes, and specifying the compression algorithm to be used. Othermessage types may include encryption message types as well. In yet otherembodiments, network application traffic management unit 200 may store afirmware image for network processing unit 300 and interact (typicallyduring initialization) to determine the firmware image stored on networkprocessing unit 300. Network application traffic management unit 200, ifit determines that a firmware update is required, may transmit thefirmware image to network processing unit 300 in one to a plurality ofcontrol messages.

B.1. SizingData Message

Network application traffic management unit 200 transmits a SizingDatamessage to provide an initial configuration to the network processingunit 300. In a particular implementation, network processing unit 300simply forwards received packets along the network path to theirdestination without processing, until it receives a configuration fromthe network application traffic management unit 200. The SizingDatamessage indicates the capacities of the Control Plane. In a particularimplementation, the Data. Plane allocates its memory to be aligned withthese capacities, such as the number of partitions, the number ofsupported traffic classes, the number of supported, flow blocks. Thefollowing illustrates an example format of a SizingData messageaccording to one particular implementation of the invention. In aparticular implementation, objects, such as data flows, partitions, andclasses are referenced relative to an Index and an instance identifier.

typedef struct_ObjectReference { uint16_t index; uint16_t instance; }ObjectReference; typedef struct_SizingData { uint16_t type; #definekSizingDataType 787 uint16_t seq; // sequence number uint32_t ipaddr; //main address of NATM uint32_t ptncount; // # of partitions uint32_tclasscount; // # of traffic classes uint32_t flowcount; // # of flowblocks uint16_t servicecount; // # of services uint16_ttriggerexpiration; // in minutes uint32_t triggerlines; //number of DTPhash groups } SizingData, *SizingDataPtr;

The triggerexpiration parameter indicates the length of time that acached policy decision is valid. In one implementation, a null or zerovalue indicates that the cached policy decision does not time out. Thetrigger lines parameter indicates the number of hash groups the DataPlane should allocate in memory.

B.2. PartitionInfo Message

Network application traffic management unit 200 sends PartitionInfomessages when a partition is created, deleted, moved, or resized. APartitionInfo message can also be transmitted in response to aReTransmitRequest message sent by the Data Plane (see below),

typedef struct_PartitionInfo { uint16_t type; #define kPartitionInfoType789 uint16_t seq; ObjectReference partition;   // this partition uint8_tdirection; #define kInboundDirection 0 #define kOutboundDirection 1uint8_t isroot; uint8_t action; #define kPartitionActionNew 1 #definekPartitionActionResize 2 // linkages unchanged #definekPartitionActionDetach 3 // detach, don't delete, uses old parent#define kPartitionActionReattach 4 // is detached, parent is new parent#define kPartitionActionDelete 5 // should be leaf, parent is old parent#define kPartitionActionRetransmit 6 uint8_t isdefault; ObjectReferenceparent; uint32_t minbw; uint32_t maxbw; } PartitionInfo,*PartitionInfoPtr;

Some partition attributes in the PartitionInfo message include theminimum (minbw) and maximum (maxbw) bandwidth allocated to thepartition, the identity of the parent of the partition, the direction oftraffic flow (direction) to which the partition corresponds, and whetherthe partition is the default partition (isdefault) or root (isroot) forthat direction.

B.3. FlowInfo Message

A major aspect of the control functions performed by the Control Planeis embodied in the FlowInfo message sent by the Control Plane when ithas decided what policy or policies should, be applied to a new dataflow. In one implementation, the Control Plane is operative to create adata structure for the flow, and transmit a FlowInfo message to the DataPlane. The FlowInfo message causes the Data Plane to create a flowblock, which is a data structure or object, for storing variousattributes of the data flow. The flow block is identified by a FlowIndexand an instance value. Attributes of the flow block may include one ormore of the attributes defined in the FlowInfo message set forth below.The following illustrates attributes that may be included in a FlowInfomessage according to one particular implementation of the invention.

typedef struct_FlowInfo { uint16_t type; #define kFlowInfoType 788uint16_t seq; uint32_t flowindex; // identifies flows uint16_tflowinstance; uint16_t service; //service identifier for flow uint8_tserverside; #define   kInSide   0 //server located inside #define  kOutSide   1 //server located outside uint8_t sendmore; // keepsending packets unit8_t policies[2]; #define   kPolicyPriority   0x01#define   kPolicyRate   0x86 // any of these bits #define  kPolicyPassThru   0x08 #define   kPolicyDiscard   0x10 #define  kPolicyNever   0x20 ObjectReference classes[2]; // by directionObjectReference partitions[2]; uint8_t priorities[2]; unit8_t trigger;#define   kTriggerDont   0 #define   kTriggerRemember   1 #define  kTriggerRecycled   2 #define   kTriggerServiceOnly   3 uint8_tdirection; // to hash flow attribute tuple } FlowInfo, *FlowInfoPtr;

Each data flow is identified by its FlowIndex, a number uniquelydetermined by which flow block (TCB or UCB type) was allocated to it bythe Control Plane. The FlowInfo message, in a particular implementation,contains the determined policy (for example, one of Priority, Rate,PassThru, Discard, or Never). Still further, the FlowInfo messages mayalso include a service parameter which is a value that maps to a networkapplication type (such as Oracle® database, FTP, Citrix®, HTTP, andother network applications). The serverside parameter indicates whetherthe location of the server of the data flow relative to the Data andControl plane. A server is typically the host that received the initialpacket of the data flow (such as a TCP SYN) from a client host. Theinside or outside server determination is based on detection of theinitial packets of a data flow and their direction. With reference toFIG. 1, an “inside” server relative to network application trafficmanagement device 30 associated with network 40 a, is a host connectedto network 40 a, while an outside server host is a host located, acrossnetwork 50. In one particular implementation, there are policies foreach direction (“inbound” and “outbound”) or “half-flow” of the trafficflow. There are also two traffic class indices, partition numbers, andpriorities in the FlowInfo message.

The FlowInfo message may also contains control variables related tointeraction between the Control Plane and Data Plane relative to thedata flow. For example, tire Control Plane may set the sendmore variableto false to indicate that the Data Plane should, completely take overhandling packets of the data flow. For example, as described in moredetail below, the Data Plane will continue to sample packets of a dataflow to the Control Plane until it receives a FlowInfo message for thatdata flow, where the sendmore variable is set to “false.” If thesendmore variable is set to true, the Data Plane will continue to samplepackets to the Control Plane until the Control Plane transmits anotherFlowInfo message with sendmore set to false. In a particularimplementation, when packet sampling stops for a given data flow isdefined by the Control Plane, which can use this mechanism to implementone or more value added features, such as packet capture. For example,if a data flow hits a traffic class with packet capture enabled, theControl Plane can set sendmore to true and never clear it for the lifeof the data flow. Anything that required the Control Plane to handle allthe packets of a flow could be handled in this manner.

FlowInfo messages may also contain a trigger parameter indicatingwhether the Control Plane should cache the service identification(service), traffic classification (classes) and policy (partitions,priorities) decisions contained in the FlowInfo message. For example, ifthe trigger parameter is 0, the Data Plane does not cache thisinformation. If the trigger parameter is 1, the Data Plane caches thisinformation for use in connection with subsequent, data flows that matchthe server-side attributes of the current data flow. Furthermore, theControl Plane may set the trigger parameter to 2, indicating that theControl Plane acknowledges the matching of the data flow to cachedinformation and should not be cached another time. The Control Plane mayalso set the trigger parameter to 3 to indicate that the flowinformation (such as network application type) should be cached in thetriggering cache, but that default policies (instead of cached policiesin the triggering) should be applied. Processing of FlowInfo messages isdescribed in more detail below.

In a particular implementation, FlowInfo messages have the same IP andTCP/UDP protocol headers as the data flow to which they refer. In such aconfiguration, the network processing unit 300 computes the same hashvalue for the 5-tuple (see below) of header attributes that are used toidentify data flows. Network processing unit 800 has functionalitiesthat allow for the packets of the same data flow to be processed by acommon processor core. Addressing the FlowInfo messages in this mannerallows the control messages for a flow to be processed by the sameprocessor core handling data packets of the flow. Alternatively, theattributes of the 5-tuple for the data flow can also be included in theFlowInfo message, and the addresses in the headers can correspond to theaddresses of the Data Plane and Control Plane.

B.4. OverloadStatus and BadHostInfo Messages

The Control Plane uses the OverloadStatus and BadHostInfo messages tocontrol the flow of sampled packets from the Data Plane. The followingdefines the formats of the OverloadStatus and BadHostInfo messagesaccording to an implementation of the invention.

typedef struct_OverloadStatus { uint16_t type; #definekOverloadStatusType 791 uint16_t seq; uint8_t overloaded; }OverloadStatus, *OverloadStatusPtr; typedef struct _BadHostInfo {uint16_t type; #define kBadHostType 792 uint16_t seq; uint32_t ipaddr;uint8_t client; uint8_t direction; } BadHostInfo, *BadHostInfoPtr;

In one implementation, the Data Plane is not configured with a “maximumrate” the Control Plane is capable of handling. Rather, the ControlPlane learns this from the OverloadStatus messages sent from the ControlPlane when it senses an overload condition, such as a threshold, numberof packets in one or more receive queues. This signaling scheme allowsthe Data Plane to automatically adjust, to interfacing with other modelsof a network application traffic management unit 200 or recognizing thatdifferent network traffic mixes may place different loads on theclassification mechanisms of the Control Plane.

In a particular implementation, the Control Plane also indicates to theData Plane when hosts are behaving badly. For example, the Control Planemay send a BadHostInfo message to inform the Data Plane of any hosts theLoad Shedding feature decides are behaving badly. The Data Plane canreduce or block traffic for a period of time in response to theBadHostInfo messages. In one implementation, the Data Plane can grow thepacket rate sampled to the Control Plane (relative to a given host)until it receives a subsequent BadHostInfo message from the ControlPlane.

B.5. ReTransmitRequest Message

As discussed above, the Data Plane may also transmit messages to theControl Plane. For example, the Data Plane may send a ReTransmitRequestmessage that lets the Data Plane ask for a replay of certain of thedownward control messages. In a particular implementation, the DataPlane may transmit a ReTransmitRequest message each time it sees anobject referenced in a control message for which it has no information.For example, the Data Plane may request a replay of the SizingDatamessage, which may get lost while the Control Plane is booting up, orthe OverloadStatus message, which might get lost in an overloadcondition, and the PartitionInfo message, which, is helpful forresynchronization when the Control Plane comes up after the Data Plane.ReTransmitRequest messages also facilitate resynchronization between theControl Plane and the Data Plane in the event of a fault or crash ofeither the Data Plane or the Control Plane. The following illustratesthe format of a ReTransmitRequest message according to one particularimplementation of the invention.

typedef struct_ReTransmitRequest { uint16_t type; #definekRetransmitType 775 uint16_t seq; uint16_t what;   // message type toresend uint16_t index; // partition index (kPartitionInfoType) }RetransmitRequest, *RetransmitRequestPtr;

B.6. MeasurementSample Message

In one implementation, the Control and Data Planes implement ameasurement data signaling scheme to allow measurement engine 140 tomaintain network statistics relative to data flows, partitions andtraffic classes. In a particular implementation, the Data Planetransmits MeasurementSample messages to the Control Plane such that itcan update the values of various statistics it maintains. The followingillustrates the format of a MeasurementSample message according to onepossible implementation of the invention.

typedef struct_MeasurementSample { ObjectReference obj; uint32_tpackets; uint32_t bytes; } MeasurementSample, *MeasurementSamplePtr;typedef struct _MeasurementData { uint16_t type; #definekMeasurementDataType 777 uint16_t seq; uint16_t count; uint8_t flavor;#define kMeasurementFlavorPartition 1 #define kMeasurementFlavorClass 2#define kMeasurementFlavorService 3 uint8_t pad; MeasurementSamplesamples[1];   //count samples   here } MeasurementData,*MeasurementDataPtr;In one implementation, the Data Plane maintains byte and packet, countsper traffic class and per partition (excluding the “sampled” packetswhich the Control Plane has already counted). On a periodic basis, abackground task will bundle up samples for active classes andpartitions, and forward the data back to the Control Plane for recordingin MeasurementSample messages.

B.7. LittleNote Message

For diagnostic purposes, the Control Plane may send log data (such as“printf” output) LittleNote messages. At the Control Plane, the eventsmay be logged into the Control Plane “system event” log as well ascopied to any configured syslog servers. The following illustrates theformat of a LittleNote message according to one possible implementationof the invention.

typedef struct_LittleNote { uint16_t type; #define kLittleNoteType 779uint16_t seq; uint8_t level; // SYSLOG_LEVEL #define kNoteEmerg 0#define kNoteAlert 1 #define kNoteCrit 2 #define kNoteErr 3 #definekNoteWarning 4 #define kNoteNotice 5 #define kNoteInfo 6 #definekNoteDebug 7 uint8_t pad; uint16_t reserved; char note[1]; // nullterminated } LittleNote, *LittleNotePtr;

C. Example Process Flows

FIG. 5A illustrates an example process flow, according to one possibleimplementation of the invention, executed by the network processing unit300. When network processing unit 300 receives a packet (502), thehardware of the network processing unit 300 includes a packet parsinglogic circuit that parses a received packet and computes a hash of anattribute tuple of the received packet. In one implementation, the tuplecomprises the source IP address, destination IP address, source portnumber, destination port number, and a protocol identifier (such as TCP,UDP, etc.). Network processing unit 300 uses this 5-tuple to identifydifferent data flows between hosts. In one implementation, the values ofthese fields are arranged in an order, depending on the direction thepacket is traveling, to ensure that the resulting hash is the same fordata flows of the packet transmitted in either direction. In oneimplementation, the hardware-computed hash is a 16-bit hash. A secondarylonger (e.g., 32-bit) hash, or a hash using a different algorithm, ofthe same 5-tuple is also computed to identify the data flow. In oneimplementation, the 16-bit hash computed by hardware may map to one ormore secondary hashes. To identify a data flow, the 16-bit hashessentially narrows the search space to a subset of the secondary hashesthat are mapped to the 16 bit hash.

As FIG. 5A illustrates, if the received packet is not an IP packet(504), network processing unit 300 forwards the packet along toward itsdestination from an egress interface that corresponds to the packetinterface on which the packet was received (506). In otherimplementations, network processing unit 300 and the control plane canbe configured to process non-IP packets as well. Furthermore, if thereceived packet is a control message (see above) from the Control Plane(508), network processing unit 300 passes the control message to aprocess that programs the data plane by changing one or more attributesof the data plane configuration (510). For example, network processingunit 300 may create a flow block in response to a FlowInfo message. Inone implementation, flow blocks are identified by the FlowIndex valuesof FlowInfo messages. See also FIG. 5C, discussed below. Otherwise,network processing unit 300 determines whether it has a flow referencethat matches the hash it previously computed for the packet (512). Ifnot, network processing unit 300 creates a flow reference in response tothe data packet (514). A flow reference includes a key (typically a hashof the 5-tuple attribute values, see above), a FlowIndex value (indexinginto an array or table (or other data structure) of flow blocks, and aflow instance identifier. When initially created, the flow referenceincludes a null FlowIndex value, which may subsequently be modified toindex to a flow block when created. As discussed in connection with FIG.5B, however, the FlowIndex value may be set to an entry of cached flowinformation in a triggering cache.

As FIG. 5A illustrates, network processing unit 300 may be configured tosample packets to network application traffic management unit 200 (518),if there is no flow block created for the data flow (516). If there isno flow block for the data flow, network processing unit 300 may applyone or more default policies to the packet (520). For example, networkprocessing unit 300 may assign the packet to a default partition havinga maximum bandwidth parameter enforced by scheduler 318. If a flow blockexists (516), network processing unit 300 determines whether to samplethe packet to network application traffic management unit 200 (522,524). For example, the sendmore attribute of the flow block may be setto true. In addition, the received packet may be a data flow orconnection-terminating packet (such as a TCP FIN or RST). In aparticular implementation, network processing unit 300 is configured totransmit connection-initiating (e.g., handshake or TCP SYNs andSYN/ACKs), and connection-terminating packets (e.g., TCP FINs, RSTs,etc.) to allow network application traffic management unit 200 to set upand tear down data structures as required. Lastly, as FIG. 5Aillustrates, network processing unit 300 applies one or more policiesidentified in the flow block to the packet (526). For example, the flowblock may identify a partition, or a priority policy. The flow block mayalso identify other policy types, such as a diffserv or tagging policy.

In one implementation, the internal processes of network applicationtraffic management unit 200 assume that a data flow has terminated if apacket associated with the data flow has not been encountered in athreshold period of time. Termination of a data flow may cause thenetwork application traffic management unit 200 to tear down variousdata structures for the data flow (to allow the memory space to be used,for other data flows). In such implementations, the network processingunit 300 may be configured to periodically sample packets to networkapplication, traffic management unit 200 (even, after sendmore has beenset to false) to ensure that the network application traffic managementunit 200 does not deem the flow terminated. The rate at which thesepackets are sampled will depend on the configuration of the networkapplication traffic management unit 200 and the threshold values it usesto deem flow terminated. In such an implementation, the decisional logicrepresented in 522 of FIG. 5A can be augmented to include a check thatcompares the last sample time to the current time and to conditionallysample the packet if the time difference is greater than a threshold.

C.1. Server-Side Triggered Policies

According to the process flow illustrated in FIG. 5A, the Data Planeapplies one or more default policies to new data flows until it receivesa FlowInfo message from the Control Plane identifying one or morepolicies to be applied. FIGS. 5B and 5C illustrate an alternativeimplementation where the Data Plane may possibly apply cached policies,as opposed to default policies, to new data flows that match one or morecriterion, such as server-side related attributes of IP address and portnumber.

As discussed above, the Data Plane may maintain a set of flow objects ina reserved, memory space. The flow objects include flow attributeinformation and one or more applicable policies (see above). The DataPlane may also reserve memory (a triggering cache) for the caching offlow attribute information for possible re-use for subsequent dataflows. As discussed above, the Data Plane may store in a triggeringcache certain flow information responsive to the value of the triggerparameter in FlowInfo messages. FIG. 5C illustrates an example processthat a Data Plane may apply to a received FlowInfo message. As FIG. 5Cshows, when the Data Plane receives a FlowInfo message, it maps theFlowIndex value in the FlowInfo message to a flow object entry in thereserved memory space (580), and saves the flow configuration data inthe Flow Block entry (562). If the trigger parameter is set to“remember” (564), the Data Plane then accesses the triggering cache toidentify whether a matching entry exists (586). A cache entry maycomprise a key value, a time stamp, and flow attribute and configurationinformation (such as service identifiers, policies and the like). Thekey comprises a tuple of an IP address, port number and a protocolidentifier. In one implementation, the IP address and port number usedfor the key is determined with reference to the serverside parameter inthe FlowInfo message. That is, the Data Plane uses the serversideparameter to identify the IP address and port number of the server ofthe data flow and uses this information and the protocol identifier togenerate the key.

In one implementation, the cache is organized as a set of hash groups,where each hash group includes a set of N cache entries. In addition,each cache entry is also addressed relative to an index value that mapsto the memory space reserved for the cache. To identify a possiblematching entry, the Data Plane may compute a hash of the key to select ahash group, and then search (possibly computing a secondary hash) theentries for a key that matches. If a matching entry in the cache exists,the Data Plane updates or replaces the flow configuration information inthe cache entry and updates the time stamp (568). If no matching entryexists, the Data Plane selects a cache entry within the hash group tostore the flow configuration data (570), setting a time stamp as well.If there is no empty slot in the hash group, the Data Plane may use aleast-recently used (LRU) algorithm to select an entry based on thevalue of the time stamps.

The cached flow information may be used for subsequent data flows, wherethe server-side attributes of the flow match the cached information. Inother words, use of the cached flow information may be triggered by thedestination (server-side) IP address, destination (server-side) portnumber and protocol identifier of the initial packet of the data flow.How the Control Plane sets the trigger value of the FlowInfo message mayalso be specified by one or more policies. The configuration ofserver-side triggered caching policies can be based on a variety offactors, and can be fully automated, partially automated based on a ruleset, and/or manually performed by a network administrator. For example,a network administrator may have configured a traffic classificationbased on a network application where the server port is static. TheControl Plane may set the triggering policy to “remember” (1) bydefault. In addition, a network administrator may have configured atraffic class with a matching rule based on a host list. In oneimplementation, the Control Plane might, as a default operation, wantthe Data Plane to cache the partition and policies for a server foundthe user-created host list. The traffic classification database of theControl Plane may be configured to associate server-side trigger policesto various network application types. Application of the server-sidetriggered caching policies can be based on a variety of factorsassociated with, the behavior of the network application or service typeand/or other factors, such as the configuration of the networkapplication traffic management device 30. For example, the Control Planemay implement a default rule where server-side triggered policies areimplemented for all traffic classifications, except where the trafficclassifications are based on network applications that have certainspecified behaviors. For example, network applications where data flowsuse dynamic port number assignments, as FTP data flows, may not besuitable candidates for caching sewer-side triggered policy information.However, information relevant to FTP control flows can be cached sincethe server-side port for the control flow is generally static. Inaddition, a traffic classification database may store a traffic classbased on known network applications, such as YouTube, MySpace, amongothers. Recognizing that the IP address and port number of the siteshosting such network applications is not likely to be dynamic, aserver-side trigger policy may be used for the traffic class that causesthe trigger value to be set to “remember” when data flows hit thattraffic class. Accordingly, packets of subsequent data flows sourcedfrom a client host initiating a connection to a video sharing site atthe IP address and port number will hit the cache, causing the cachedpolicies to be applied to the data flows. In addition, server-sidetriggered caching policies can also be based on whether the networkadministrator has specified a default policy for a given traffic class,and an exception list (based on client IP address or other attribute).Use of an exception list may indicate that flow information caching maynot be suitable. Still further, traffic classification can also be basedon attributes that are orthogonal to server-side attributes, such asdiffserv or DSCP markings. The Control Plane may be configured to setthe trigger policy to “don't” (0) or “service-only” (3) for such trafficclasses.

FIG. 5B illustrates a process flow that may be implemented by the DataPlane in connection with, server-side triggered policies. Theillustrated process is quite similar to that described in connectionwith FIG. 5A. However, in FIG. 5B, after the Data Plane creates a flowreference, it accesses the triggering cache to determine if a matchingcache entry exists (550). To find a match, the Data Plane generates akey comprising the server-side IP address, server-side port number andthe protocol identifier contained in the received packet. As discussedabove, the Data Plane may hash this key value to identify a hash group,and then search the entries of the hash group to find a matching cacheentry. Additionally, a cache hit may also be conditioned on the value ofthe time stamp in the cache entry relative to a time out value(triggerexpiration) transmitted by the Control Plane in a SizingDatacontrol message (see above). If no matching entry is found, the DataPlane operates as discussed above relative to the data flow. If amatching entry is found, however, the Data Plane sets the Flow Indexvalue in the flow reference to the index value (K) of the matching cacheentry, and sets a flag associated with the packet that causes the dataplane to insert a header when sampling the packet to the control plane(see FIG. 8, and accompanying description) (552). Accordingly, when theprocess proceeds to step 516, the flow reference identifies a flow indexvalue, causing the cached policies in the cache entry to be applied(526), as opposed to the default policy (520). However, if the triggerparameter has been set to 3 (ServiceOnly), as discussed above, then theData Plane applies the default policy. This configuration allows theData Plane to apply desired or more appropriate policies to the dataflow from the first packet, as opposed to later in the data flow. TheData Plane will also continue to sample the packets until a FlowInfomessage for the flow is received. Other implementations are possible.For example, after setting the packet, flag and flow index value (552),the process flow may proceed directly to sampling the packet (524).

C.2. Packet Sampling

FIGS. 6, 7 and 8 set forth process flows, according to one particularimplementation of the invention, directed to sampling packets to networkapplication traffic management unit 200. FIG. 6 is a state diagramshowing when, and the aggregate rate at which, packets across all dataflows are sampled to network application traffic management unit 200.FIG. 7 illustrates a process implemented by the control plane whensampling packets to network application traffic management unit 200.

As FIG. 6 provides, network processing unit 300 is configured to samplepackets to network application traffic management unit 200 at a packetsampling rate (602). If the number of packets to be sampled would exceedthe current packet sampling rate, network processing unit 300 samples asubset of the packets. For example, if network processing unit 300receives during a given interval 100 packets to be sampled per second,and the current sampling rate is 50 packets per second, networkprocessing unit 300 would not sample 50 of the packets in a one-secondinterval. Selection of which packets to sample or not can be based onone or more sampling policies, the details of which are beyond the scopeof this disclosure. As FIG. 6 illustrates, as long as the networkprocessing unit 300 does not receive an OverloadStatus messageindicating that network application traffic management unit 200 isoverloaded (604), it continues sampling packets at the packet samplingrate, periodically growing the packet sampling rate (606, 608). However,if network processing unit 300 receives an OverloadStatus messageindicating that network application traffic management unit 200 isoverloaded (604), it stops sampling packets to network applicationtraffic management unit 200 and decreases the packet sampling rate (inone implementation, by half of the current sampling rate) (610). As FIG.6 shows, network processing unit 300 discontinues packet sampling untilit receives an OverloadStatus message indicating that networkapplication traffic management unit 200 is ready to receive packets(612). In the implementation shown, network processing unit 300 may alsotransmit, a ReTransmitRequest (616) asking for an OverloadStatus messageat periodic intervals (614).

As illustrated in FIG. 7, network processing unit 300 may be configuredto conditionally sample packets to network application trafficmanagement unit 200. For example, if a host identified in the packet isidentified as a bad host in a BadHostInfo message (702), networkprocessing unit 300 may drop the packet, entirely (703), or pass thepacket to a process that decides whether to drop the packet or pass thepacket through. Furthermore, if network application traffic managementunit 200 is currently overloaded (704), network processing unit 300 doesnot sample the packet. Otherwise, network processing unit 300 samplesthe packet to network application traffic management unit 200 (706), andresets a last sample time value of the flow block. The last sample timevalue can be used to control sample of packets, such as the leaking ofpackets to network application traffic management unit 200, see above.As discussed above, in one implementation, network processing unit 300samples the packet by transmitting it to network application trafficmanagement unit 200. Network processing unit 300 then resumes processingof the packet, such as applying one or more policies.

FIG. 8 illustrates an example process directed to sampling packets tothe Control Plane, where the Data Plane constructs and inserts serviceidentifying headers into the first packet of a flow. Referring to FIGS.7 and 8, when the Data Plane samples a packet to the control plane(706), it determines whether the packet of the data flow matches a cacheentry (802), and whether the packet flag has been set (803). If so, theData Plane constructs a header including a service identifier containedin the matching cache entry, and inserts the header into the sampledpacket (804). In a particular implementation, the header is a 4-byteheader comprising a 16-bit operation code identifying a service hinttype, and a 16-bit value of the service identifier. In oneimplementation, the header is inserted after the MAC layer header, whereVLAN tags are normally inserted. The Data Plane then sends the sampledpacket to the Control Plane (806), resetting a last sample packet time.When the network interface returns an indication that the sampled,packet has been sent (808), the Data Plane, rather than deleting thepacket from the buffer, passes the packet to a policy enforcementmechanism (812), removing the header if one has been added (809, 810).

FIG. 9 shows a process that a Control Plane may apply to sampledpackets. As FIG. 9 illustrates, when the Control Plane receives asampled packet (852), it scans for a header including the service hintoperation code (854). If a header exists, the Control Plane extracts theservice identifier from the header and adds it to a data structure, suchas a flow object, for the data flow (856), and passes the sampled packetto the classification mechanism (858). The existence of a serviceidentifier in the flow object indicates to the classification mechanismthat service type identification for the data flow has beenaccomplished. This saves computing resources for other tasks, since theControl Plane need not process the data flow to identify a service typeand can proceed to classifying the data flow,

In the implementation described above, a large portion of the networkprocessing is offloaded to the network processing unit 300, which withits dedicated hardware-level processing features allows for fasterprocessing of network traffic. In the implementation described above,the network processing unit 300 handles network traffic usingpre-existing programming. If it does not have a record of a flow and itsclass, policy, or partition (or matching cached, information), itapplies defaults to the traffic, and samples the traffic to the networkapplication traffic management unit 200. In this manner, the performancerequirements on the network traffic management unit 200 aresignificantly reduced since it sees only a limited subset of the traffic(typically, the initial packets, one or more leaked packets to preventflow termination processes of the Control Plane, and possiblyterminating packets of a data flow). The network application trafficmanagement unit 200 can classify the traffic fully and report back theclass, partition, and policy of the data flow when it is done. In themeantime, the network processing unit 300 continues to use defaults orserver-side triggered cached policies until it receives programming forthe specific flow. Once programming is received, it handles the trafficusing the policies specified by the network application trafficmanagement unit 200. Further, the use of server-side triggered cachepolicies provides additional advantages. The major work done by theControl. Plane is classifying new flows. Another way to offload morecycles from it is to recognize that once the Control Plane has carefullyexamined a new flow and decided it is a given traffic class, it islikely that the next new flow to the same server IP address and port isalso likely to be the same traffic class. Because the Data Plane cachesthis possibly expensive determination and applies it to subsequent newflows (within a reasonable time) made to the same port on the sameserver host, much duplicate work is eliminated from the Control Plane,allowing it to handle more new flows per minute. Furthermore,server-side triggered policies also facilitates the application ofpolicies and other operations, such as tunneling and protocolacceleration, where it is desirable to begin with the first packet ofthe data flow.

The present invention has been explained with reference to specificembodiments. For example, the functions performed by network processingunit 300 can be extended to include compression and network accelerationtechnologies. For example, network processor units may havehardware-based compression on chip. In such an implementation, networkprocessing unit 300 can be configured to forward all tunnel discovery,set up and management messages to network application traffic managementunit 200 which, processes the messages and transmits tunnel controlmessages to network processing unit 300. The tunnel control messages mayspecify the IP address of the tunnel endpoint, the compression algorithmto use, and the like. In such an implementation, the FlowInfo messagescan be extended to identify which tunnel the packets of the data floware to be placed. Still further, in some implementations, the controlplane may be implemented by one or more cores of a multi-core processor,while the data plane may be implemented by one or more remaining coresof the multi-core processor. In other implementations, the control planeand data plane can be implemented on the same physical host but onseparate virtual machines. Still further, the server-side triggeredpolicy caching mechanisms described herein may be used, in other systemarchitectures where there is no physical or logical separation betweencontrol and data planes. For example, in one implementation, server-sidetriggered policy mechanisms can be used to bypass service typeidentification and or traffic classification mechanisms that wouldotherwise be applied to new flows. Other embodiments will be evident tothose of ordinary skill in the art. It is therefore not intended thatthe present invention be limited, except as indicated by the appendedclaims.

1. An apparatus comprising a control plane; and a network processingunit operative to: selectively sample received packets of the respectivedata flows to the control plane; maintain a flow database of flowinformation entries, each comprising one or more policies and indexed byone or more data flow attributes; maintain a triggering cache of flowinformation entries, each comprising one or more cached policies andindexed by one or more server-side attributes; match packets ofrespective data flows to flow information entries in the flow database;else match packets of new respective data flows to flow informationentries in the triggering cache based on the one or more server-sideattributes; apply the one or more policies to received packets ofrespective data flows that match corresponding flow information entriesin the triggering cache or tire flow database, otherwise apply one ormore default policies to received packets of respective data flowsaccording to a current data plane configuration; wherein the controlplane is operative to process packets sampled by the network processingunit; and transmit control messages to the network processing unitidentifying one or more policies for the respective data flows; whereinone or more of the control messages include indications operable tocause the data plane to add flow information entries in the flowdatabase, and store the one or more policies in the triggering cache inassociation with the one or more server-side attributes.
 2. Theapparatus of claim 1 wherein the one or more sewer-side attributescomprise an IP address and a port number.
 3. The apparatus of claim 1wherein the one or more data flow attributes of the flow databasecomprise client-side and server-side attributes.
 4. The apparatus ofclaim 1 wherein one or more of the control messages include indicationsoperable to cause the data plane to store network applicationidentifiers in the triggering cache in association with the one or moreserver-side attributes.
 5. The apparatus of claim 4 wherein one or moreof the control messages include indications operable to cause the dataplane to use the default policy for data flows matching the server-sideattributes in the control messages.
 6. The apparatus of claim 4 whereinthe data plane is further operative to transmit to the control plane theone or more network application identifiers stored in the triggeringcache in response to data flows matched to corresponding flowinformation entries in the triggering cache.
 7. The apparatus of claim 2wherein the one or more policies is further associated with one or moreadditional attributes.
 8. The apparatus of claim 7 wherein the one ormore additional attributes comprise a protocol identifier.
 9. Theapparatus of claim 1 wherein the network processing unit is operative tosample connection-initiating packets of respective data flows to thecontrol plane.
 10. The apparatus of claim 1 wherein the networkprocessing unit is operative to sample the initial packets of respectivedata flows to the control plane.
 11. The apparatus of claim 1 whereinthe network processing unit is operative to sampleconnection-terminating packets of respective data flows to the controlplane.
 12. The apparatus of claim 1 wherein the network processing unitis operative to periodically sample packets of respective data flows tothe control plane.
 13. The apparatus of claim 1 wherein the controlplane is operative to classify the respective data flows based on thesampled, packets: identify one or more policies for the respective dataflows; and provide the identified policies for the respective data flowsin the control messages transmitted to the network processing unit. 14.The apparatus of claim 13 wherein the control plane is operative todetermine whether one or more policies in respective control messagesare to be cached.
 15. The apparatus of claim 1 wherein the control planeis a network application traffic management unit.
 16. The apparatus ofclaim 1 wherein the control plane is operative to transmit controlmessages configured to control sampling of received, packets of dataflows by the network processing unit.
 17. The apparatus of claim 1wherein the one or more policies include partition identifiers forrespective data, flows.
 18. The apparatus of claim 1 wherein the controlplane is further operative to transmit control messages indicating anoverload condition; and wherein the network processing unit is operativeto discontinue sampling of received packets responsive to a controlmessage indicating an overload condition.
 19. The apparatus of claim 1wherein the control plane is further operative to transmit controlmessages indicating an overload condition; and wherein the networkprocessing unit is operative to sample packets to the control plane at afirst sampling rate; discontinue sampling of received packets responsiveto a control message in (Heating an overload condition; and resumesampling of packets at a sampling rate less than the first samplingrate.
 20. The apparatus of claim 19 wherein the network processing unitis further operative to increase the sampling rate until receipt of acontrol message indicating an overload condition
 21. The apparatus ofclaim 1 wherein the control plane is further operative to transmitcontrol messages identifying a bad host; and wherein the networkprocessing unit is operative to selectively discard received packetsassociated with the bad host.
 22. The apparatus of claim 1 wherein thecontrol plane is housed in a first enclosure, and the network processingunit is external to the first enclosure.
 23. The apparatus of claim 1wherein the control plane is operative to transmit retransmissionrequest messages identifying one or more objects referenced in controlmessages transmitted from the control plane for which it has noconfiguration information.
 24. An apparatus comprising a control plane;and a network processing unit operative to; selectively sample receivedpackets of the respective data flows to the control plane; maintain aflow database of flow information entries each indexed by one or moredata flow attributes; maintain a triggering cache of flow informationentries, each comprising one or more cached network applicationidentifiers and indexed by one or more server-side attributes; matchpackets of respective data flows to flow information entries in the flowdatabase; else match packets of new respective data flows to flowinformation entries in the triggering cache based on the one or moreserver-side attributes; conditionally attach network applicationidentifiers in the sampled packets responsive to received packets ofrespective data flows that match corresponding flow information entriesin the triggering cache; wherein the control plane is operative toprocess packets sampled by the network processing unit; and transmitcontrol messages to the network processing unit including one or morenetwork application identifiers for the respective data flows; whereinone or more of the control messages include indications operable tocause the data plane to add flow information entries in the flowdatabase, and store the one or more the network application identifiersin the triggering cache in association with the one or more server-sideattributes.
 25. An apparatus, comprising a memory; one or moreprocessors; one or more network interfaces; and a firmware applicationcomprising instructions operative to cause the one or more processorsto: selectively sample received packets of the respective data flows toa control plane; maintain a flow database of flow information entries,each comprising one or more policies and indexed by one or more dataflow attributes; maintain a triggering cache of flow informationentries, each comprising one or more cached policies and indexed by oneor more server-side attributes; match packets of respective data flowsto flow information entries in the flow database; else match packets ofthe respective data flows to flow information entries in the triggeringcache based on the one or more server-side attributes; apply the one ormore cached policies to respective data flows that match correspondingflow information entries in the triggering cache, otherwise apply one ormore policies to received packets of respective data flows according toa current data plane configuration; change the data plane configurationresponsive to control messages received from the control planeidentifying one or more policies to be applied to respective data flows;and responsive to control messages including caching indications, storethe one or more policies in the triggering cache in association with thevalues of the one or more server-side attributes.
 26. The apparatus ofclaim 25 wherein the apparatus further comprises a packet parsing logiccircuit operative to parse received packets into one or more attributevalues.
 27. The apparatus of claim 25 wherein the firmware is operativeto sample connection-initiating packets of respective data flows to thecontrol plane.
 28. The apparatus of claim 25 wherein the firmware isoperative to sample connection-terminating packets of respective dataflows to the control plane.
 29. The apparatus of claim 25 wherein thefirmware is operative to periodically sample packets of respective dataflows to the control plane.
 30. An apparatus comprising a memory; one ormore processors; one or more network interfaces; and a control planeapplication, stored in the memory, comprising instructions operative tocause the one or more processors to: receive one or more sampled packetsof respective data flows from a network processing unit; classify therespective data flows based on the sampled, packets; identify one ormore policies for the respective data flows; and transmit controlmessages including the identified policies for the respective data flowsto the network processing unit, wherein one or more of the controlmessages include indications operable to cause the network processingunit to cache the one or more policies in a triggering cache inassociation with one or more server-side attributes.
 31. The apparatusof claim 30 wherein the control plane application is operative toidentify network applications for the respective data flows, and whereinone or more of the control messages include indications operable tocause the data plane to store network application identifiers in thetriggering cache in association with the one or more server-sideattributes.
 32. The apparatus of claim 31 wherein one or more of thecontrol messages is further operative to cause the data plane to returna network application identifier for sampled packets matching thetriggering cache, but to apply a default policy to the packets.
 33. Amethod comprising receiving one or more packets of a first data flow,wherein the first data flow is identified relative to one or moreclient, attributes and one or more server attributes; matching the firstdata flow to a traffic classification, wherein the trafficclassification maps to one or more policies; storing the server sideattributes of the first data, flow and the one or more policies in atriggering cache; and applying the one or more policies in thetriggering cache to subsequent data flows having server attributes thatmatch the server attributes of the first data flow.